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ABSTRACT 


Interim  Progress  Report,  1  Feb  1988  -  31  Jan  1989 

Grant  AFOSR  88-0140 

I 

Visual  Motion  Perception  V 
George  Sperling,  New  York  University 


The  main  activities  throughout  this  grant  have  been  carrying  out  the  experimental  research  set  forth 
in  the  proposal  (1987),  following  up  promising  leads  that  developed  in  the  course  of  this  woik,  and 
preparing  manuscripts  for  publication.  The  work  is  best  describe  by  the  publications  and  technical 
reports;  these  are  appended.  A  brief  overview  is  (vovided  below. 


FACIUTIES 

A  highly  versatile  laboratory  has  been  created  for  research  in  almost  any  area  of  vision  or  cognition 
as  described  in  previous  progress  reports  and  the  current  proposal. 


PERSONNEL 

Principle  investigator,  George  Sperling,  Professor  of  Psychology  and  director  of  the  Human 
Information  Processing  Laboratory,  20%  time  (averaged  over  12  months).  The  pronoun  we  is  used  in  this 
report  to  refer  to  the  PI  in  conjunction  with  one  or  more  of  the  other  investigators  and  staff. 

Full-time;  Research  Associate,  Dr.  Charles  Chubb,  worked  primarily  on  visual  motion  and  on  related 
mathematical  issues. 

Systems  Programmer,  Dr.  Penelope  Hall,  who  was  visiting  NYU  from  Clambridge,  England,  on 
limited-time  appointment,  completed  her  tour  of  duty  during  the  year.  It  was  not  possible  to  find  a  suitable 
replacement  during  the  grant  year. 

Part-time:  Dr.  Anne  Sutter,  post-doctoral  fellow  supported  by  mainly  by  NEI  (and  not  at  all  by 
USAF)  joined  the  HIP  Laboratory  in  September,  1988.  In  the  part  of  her  work  relevant  to  the  current  AF 
project,  she  set  up  and  began  running  an  experiment  comparing  perceptual  analysis  of  second-order  motion 
and  texture  stimuli. 

Dr.  Barbara  Dosher,  a  consultant  who  collaborated  on  projects  in  motion  p^eption. 

Karl  Gegenfurtner,  a  graduate  student,  who  worked  paijly  on  attention  during  this  period  (suppcxted 
by  ONR)  and  partly  on  visual  projects  related  to  color  processing  with  John  Krauskopf. 

Roman  Yangarber,  a  part-time  student  programmer/research  associate  worked  on  this  grant  for  the 
first  half  of  the  grant  period.  He  participated  in  developing  and  programming  a  structure  from  motion 
algorithm,  a  project  for  which  he  still  owes  a  report. 

Patrick  Whelan,  a  part-time  student  administrative  assistant  who  did  accounting  and  related  tasks, 
resigned  during  the  grant  period  and  was  replaced  by  Pamela  Stark.  Ms.  Stark  has  a  B.S.  in  biology  and 
splits  her  time  between  serving  as  administrative  assistant  on  this  project,  on  an  ONR  project  for  the  same 
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PI,  and  her  own  continuing  graduate  work.  4^  f.  O^fl  •  ^  q 

RESEARCH  OVERVIEW  ^  ^ 

The  motion  projects  described  in  this  report  include  mainly  the  specification  of  nonFourier  second- 
order  motion  detection  systems,  algorithms  and  issues  in  deriving  structure  from  motion  and  the 
psychophysical  basis  for  the  kinetic  depth  effect,  spatial  interactions  in  low-level  vision,  and  secondarily, 
issues  in  ideal  detection  and  decision  theory,  and  the  completion  of  previously  initiated  work  in  object 
recognition. 

Progress  is  detailed  in  the  papers  and  manuscripts  impended  to  this  report.  A  bri^  survey  of  the  work  on 
nonFourier  motion  perception  is  provided  below. 

NonFourier  Motion  and  Texture  Perception. 

Distance-driven  reversal  of  apparent  motion.  We  studied  a  collection  of  stimuli  designed  to  elicit 
nonFourier  motion  in  one  direction  and  Fourier  motion  in  the  opposite  direction.  Typically  these  stimuli 
display  nonFourier  motion  when  viewed  fovially  from  short  or  moderate  viewing  ^stances,  but  reverse 
direction  and  display  Fourier  motion  when  view^  either  peripherally  or  fiom  far  away.  These  stimuli  are 
extremely  useful  in  placing  constraints  on  the  nonlinear  transformations  mediating  nonFourier  motion 
perception.  Indeed,  one  of  these  stimuli  establishes  conclusively  that  at  least  one  nonFourier  motion 
mechanism  is  mediated  by  a  full-wave  rectifying  transformation  as  opposed  to  half-wave  rectification. 

Texture  quilts.  We  extended  the  theory  of  microbalanced  random  stimuli  to  allow  us  to  study  motion 
carried  exclusively  by  spatiotemporal  modulation  of  spatial  texture  characteristics  without  engaging 
systems  processing  the  visual  sig^  for  other  types  of  motitm  information.  We  developed  methods  for 
constructing  stimuli  called  texture  quilts  whose  motion  catmot  be  seen  by  either  (i)  standard  motion 
mechanisms  (such  as  Reichardt  detectors)  or  (ii)  mechanisms  sensitive  to  motion  carried  by  any  purely 
temporal  transformation  of  the  visual  stimulus.  Titus,  for  instance,  texture  quilts  do  not  systematically 
stimulate  whatever  system  perceives  motion  carried  by  spatiotemporally  modulating  the  flicker  rate  of 
spatial  visual  noise.  We  have  found  that  it  is  quite  easy  to  construct  texture  quilts  that  display  decisive 
motion.  Spatiotemporal  modulation  of  either  the  orientation  or  the  spatial  frequency  of  spatial  texture  can 
carry  vivid  motion. 

The  dependency  of  lightness  on  texture  interactions.  We  have  discovered  that  the  perceived  contrast 
of  a  patch  of  texture  depends  substantially  on  the  contrast  of  surrounding  texture:  Indeed,  if  the 
surrounding  texture  is  modulated  in  time  between  a  mean-luminant  grey  field  and  a  field  of  high-contrast 
visual  noise,  a  disc  of  visual  noise  of  constant  contrast  situated  in  the  middle  of  the  modulating  background 
appears  to  undergo  a  large  modulation  in  antiphase  to  the  background.  We  demonstrated  that  no  such 
apparent  modulation  is  induced  if  (i)  the  surround  and  disc  are  presented  in  different  eyes,  or  (ii)  the  texture 
in  the  disc  is  different  in  spatial  frequency  content  from  the  surrounding  texture. 
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Publications  and  Papers  in  Press 


1988  *George  Sperling  and  Thomas  R.  Riedl,  Summation  and  masldng  between  spatial  frequency  bands 
in  dynamic  natural  visual  stimuli.  Investigative  Ophthalmology  and  Visual  Science,  29,  No.  3, 
ARVO  Supplement,  1988, 139.  (Abstract) 

1988  ♦Charles  Chubb  and  George  Sperling,  Processing  Stages  in  Non-Fourier  Motion  Perception. 
Investigative  Ophthalmology  and  Visual  Science,  29,  No.  3,  ARVO  Supplement,  1988,  266. 

(Abstract) 

1988  Sperling,  George.  The  magical  number  seven;  Information  processing  then  and  now.  In  William 
Hirst  (F4),  The  making  of  cognitive  science:  Essays  in  honor  of  George  A.  Milier.  Cambridge, 
UK:  Cambridge  University  Press,  1988.  Pp.  71-80. 

1988  Riedl,  Thomas  R.  and  George  Sperling.  Spatial  frequency  bands  in  complex  visual  stimuli: 
American  Sign  Language.  Journal  of  the  Optical  Society  of  America  A:  Optics  and  Image 
Science,  1988,5.  606-616. 

1988  Chubb,  Charles,  and  George  Sperling.  Drift-balanced  random  stimuli:  A  general  basis  for 
studying  non-Fourier  motion  perception.  Journal  of  the  Optical  Society  of  America  A:  Optics  and 
Image  Science,  1988,5, 1986-2006. 

1989  Chubb,  Charles,  and  George  Sperling.  Second-order  motion  perception:  Space-time  separable 
mechanisms.  Proceedings:  Workshop  on  Visual  Motion.  (March  20-22, 1989,  Irvine,  California.) 
Washington,  D.C:  IEEE  Computer  Society  Press,  1989.  Pp.  126-138. 

1989  Chubb,  Charles,  and  George  Sperling.  Two  motion  perception  mechanisms  revealed  by  distance 
driven  reversal  of  apparent  motion.  Proceedings  of  the  National  Academy  of  Sciences,  USA,  1989, 
86, 2985-2989. 

1989  Dosher,  Barbara  A.,  Michael  S.  Landy,  and  George  Sperling.  Ratings  of  kinetic  depth  in  multi-dot 
displays.  Journal  of  Experimental  Psychology:  Human  Perception  and  Performance,  1989,  15. 
(In  press,  November,  1989). 

1989  Sperling,  George,  Michael  S.  Landy,  Barbara  A.  Dosher,  and  Mark  E.  Perkins.  The  kinetic  depth 
effect  and  the  identification  of  shape.  Journal  of  Experimental  Psychology:  Human  Perception 
and  Performance,  1989, 15.  (In  press,  November,  1989). 

1989  Sperling,  George.  Three  stages  and  two  systems  of  •?•  "!  processing.  Mathematical  Studies  in 
Perception  and  Cognition,  89-1  (Rev),  New  York  U.uvc.rity,  Department  of  Psychology,  1989. 
Pp.  30,  Figs.  3.  (Accepted  for  publication  in  Prazdny  M  -^al  Issue,  Spatial  Vision,  1989.) 

1989  Barbara  A.  Dosher,  Landy,  Michael  S.,  and  George  Sperling.  Kinetic  depth  effect  and  optic  flow: 
1.  3D  shape  from  Fourier  motion.  (Accepted  for  publication  in  Vision  Research.) 

Papers  Under  Submission  for  Publication,  Technical  Reports 

1988  Parish,  David  H.  and  George  Sperling,  Object  spatial  frequencies,  retinal  spatial  frequencies,  and 
the  efficiency  of  letter  discrimination.  (Under  revision.  Vision  Research.) 

1988  Parish,  David  H.,  Sperling,  George,  and  Landy,  Michael,  S.  Intelligent  temporal  subsampling  of 
American  Sign  Language  using  event  boundaries.  Mathematical  Studies  in  Perception  and 
Cognition,  87-8,  New  York  University,  Department  of  Psychology,  1987.  Pp.  30,  Figs.  8. 
(Accepted  for  publication.  Journal  of  Experimental  Psychology:  Human  Perception  and 
Performance.) 
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1987  Farrell,  Joyce  E.,  M.  Pavel,  and  George  Sperling.  The  visible  persistence  of  stimuli  in 
stioboscq)ic  motion.  Mathematical  Studies  in  Perception  and  Cognition,  87-9,  New  York 
University,  Department  of  Psychology,  1987.  Pp.  20,  Figs.  9.  (Submitted  fw  publication.) 

1988  Spa-ling,  Geage,  and  Erich  Weichselgartner.  Movement  dynamics  of  spatial  attention. 
Mathematical  Studies  in  Perception  and  Cognition,  88-14),  New  Yoik  University,  Department  of 
Psychology,  1988.  Pp.  36,  Figs.  9. 

1988  Landy,  Michael  S..  Barbara  A.  Dosher,  George  Sperling,  and  Mark  E.  Perkins.  Kinetic  depth 
effect  and  optic  flow:  2.  Fourier  and  non-Fourier  motion.  Mathematical  Studies  in  Perception  and 
Cognition,  ^-4,  New  York  University,  Department  of  Psychology,  1988.  Pp.  45,  Figs.  6. 


George  Sperling  and  Thomas  R.  Riedl.  Summation  and  masking  between 
spatial  frequency  bands  in  dynairuc  natural  visual  stimuli  Investigative 
Opthalmology  and  Visual  Science,  1988, 29,  No.  3,  ARVO  Supplement,  139 


SUMMATION  AND  MASKING  BE'H^'EEN  SPATIAL  FREQUENCV  BANDS 
IN  DWA^^C  NATURAL  VISUAL  STIMULI 
Gcoree  Sperline  and  Thomas  R.  Riedl.  New  York  University 

D^Tiamic  images  of  a  signer  producing  individual  signs  of  American  Sign  Language 
(ASL)  were  bandpass  filiercd  in  adjacent  ^adal  frequency  bands.  Intelligibility  of  a 
band  was  determined  by  testing  deaf  subjects  fluent  in  ASL.  By  iteratively  varying  the 
center  frequencies  and  bandwidths  of  the  spatial  bandpass  Alters,  it  was  possible  to  divide 
the  original  signal  (96  x  64  pixels)  into  four  adjacent,  intelligible,  frequency  bands  with 
mean  frequencies  of  3.0. 7.5, 15,  and  25  cycles  per  frame-width.  All  bands  were  found  to 
have  the  same  temporal  frequency  spectrum  up  to  a  multiplicative  constant 

Masking  of  signals  in  band  i  by  noise  in  band  j  (4x4  conditions)  was  measured  by  a 
rating  method.  The  power  ratio  within  a  band  i,  ,  requmi  to  produce  a 

criterion  rating  response  was  the  same  for  bands  2, 3. 4  and  higher  for  band  1  (3  c/frame). 
The  logorithm  of  the  normalized  crossband  masking  effectiveness  was  inversely 
proportional  to  log  l(.7/re?„,„,  i/req^,  )l.  The  0.7  indicates  an  asymmetry:  Masking  of 
high  frequency  signals  by  low  frequency  noise  is  slightly  greater  the  masking  of  low- 
frequencies  by  highs. 

Weak  signals  from  bands  i  and  j  were  linearly  added  and  tested  for  intelligibility. 
Intelligibility  was  slightly  greater  for  signals  in  the  same  band  (i  =j)  versus  adjacent 
bands,  and  for  adjacent  bands  versus  distant  bands.  Obviously,  for  strong  signals,  adding 
different  bands  produces  roore-inteliigible  combinations  than  does  increasing  power 
within  a  band. 

The  high  intelligibility  achieved  by  the  narrow  bands  of  our  low-  resolution  signals 
indicates  that  high-resolution  broad-spccirum  signals  could  be  decomposed  into  many 
nonovcrlapping  frequency  bands,  each  of  which  contained  sufficient  information  for 
interpreting  ASL.’ 

■^Supported  in  pan  by  AFOSR  Life  Sciences  Directorate  Grant  85-0364  and  NSF  Science  and 
Technology  to  Aid  the  Handicapped,  Grant  PFR-80171 189. 
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PROCESSING  STAGES  IN  NON-FOURIER  MOTION  PERCEPTION 

Charles  Chubb  and  Ceoree  Soerline.  New  York  University 

Musi  recent  motion-perception  models  propose  detectors  that 
are  more  or  less  sharply  tuned  to  stimulus  energy  at  various  spatio- 
temporal  frequencies.  However,  it  is  easy  to  construct  random 
stimuli  which  do  not  systemmatically  excite  such  Fouher-energy 
analytic  mechanisms  and  which  nonetheless  display  strong,  consistent 
apparent  motion  across  independent  realizations  (Chubb  &  Sperling, 
ARVO,  1987V  We  show  that  two  initial  suges,  a  linear  bwdpass 
filter  followed  by  a  rectifier  (absolute  value,  square)  would  suffice  to 
expose  the  motion  information  carried  by  most  notiFourier  stimuli  to 
subsequent  Fourier-energy  analysis.  However,  we  further 
demonstrate  apparently  moving  stimuli  that  would  require  two 
successive  pairs  of  linear  filtering  and  rectification  stages  in  order  to 
be  sensed  by  a  Fourier-energy  analyzer. 

liutli  tlic  uptiiiial  spatial  frequency  and  the  sensitivity  of  the 
nonFourier  mechanism  ate  lower  than  those  of  the  Fourier-energy 
mechanism.  We  use  these  differences  to  construct  apparently  moving 
stimuli  that  grossly  violate  scale  invariance:  from  afar,  they  are  seen 
moving  in  one  direction  by  the  Fourier  mechanism;  from  close,  they 
are  seen  tnoving  in  the  opposite  direction  by  the  nonFourier 
mechanism.^ 

^van  Sanien,  J.  P.  H.  &  Sperling,  G.  J.  Opt.  Soc.  Am.  A  1985, 2,  300-321. 
^Supported  by  AFOSR  Life  Sciences  Directorate  Grant  85-0364. 


